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1.0  INTRODUCTION 

This  report  addresses  the  technical  issues  involved  in  the  analy¬ 
sis  of  the  ASETS  image  data.  Specifically,  it  outlines  issues  per¬ 
taining  to  the  alignment  of  data  from  the  CASH  instrument  collected  in 
the  ASETS  pod.  Actual  data  was  not  available  at  the  time  this  analy¬ 
sis  was  performed.  As  such,  the  topics  discussed  are  fundamental  to 
any  effort  concerned  with  correction  of  this  type  of  data. 
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2.0  FACTORS  AFFECTING  A  GEOMETRICALLY  CORRECTED  STRIP  IMAGE 

The  processing  goal  is  to  assemble  the  data  into  a  single  strip  of 
geometrically  corrected  imagery.  For  geometric  correction  to  a  fixed 
frame  of  reference  (absolute  correction),  this  entails  determining  the 
intersection  of  the  sensor's  line-of-sight  with  the  ground.  The 
parameters  necessary  for  this  determination  are  the  line-of-sight 
origin  (x.y.z),  scan  angles  (depression,  sweep),  and  the  terrain  ele¬ 
vation  (ground  z  as  a  function  of  ground  x,y).  From  this  basis  of 
first  principles,  consideration  must  be  given  to  the  various  factors 
which  affect  these  parameters  and  their  determination.  Systematic 
consideration  of  these  factors  typically  takes  the  form  of  an  error 
analysis.  The  frame  work  for  an  error  analysis  of  the  ASETS  pod  and 
CASM  combination  is  developed  in  Section  3.  From  the  eventual  error 
analysis,  informed  choices  can  be  made  regarding  measurement  accuracy 
and  whether  simplifying  approximations  are  acceptable. 

Stitching  image  frames  together  with  imperceptable  seams  presents 
several  challenges.  For  the  moment,  consider  the  situation  in  which 
no  geometric  alignment  is  required,  i.e.,  assume  perfect  alignment  and 
overlap  of  pixels.  Phenomenology  in  image  formation  can  still  result 
in  differing  values  for  many  corresponding  pixels.  The  geometry  of 
image  formation  is  perhaps  the  largest  single  contributor  to  pixel 
differences.  For  example,  obscuration  of  adjacent  objects  due  to 
vertical  displacement.  The  CASM  sensor  operating  at  200  meters  alti¬ 
tude  and  nadir  orientation  will  be  sensitive  to  vertical  displacements 
of  3.3  meters  for  objects  near  the  top  or  bottom  of  the  frame.  At  the 
same  altitude,  but  with  a  depression  angle  of  30  degrees,  only  87 
millimeter  vertical  displacement  of  adjacent  objects  results  in  a  one 
pixel  change  in  obscuration  from  the  top  to  the  bottom  of  successive 
frames.  Object  movement  could  also  be  a  factor  in  pixel  differences. 
The  CASM's  along-track  IFOV  at  200  meters  altitude  is  37  mm,  or  about 
1.5  inches  at  nadir  and  148  mm,  about  6  inches  at  a  depression  angle 
of  30  degrees.  At  a  collection  rate  of  8  frames  per  second,  objects 
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moving  faster  than  1  fps,  or  about  0.7  mph,  would  be  displaced  at 
least  1  pixel  in  the  image  data.  Given  this  sensitivity,  a  modest 
breeze  could  result  in  significant  frame  to  frame  differences  of 
objects  in  the  scene.  Finally,  scene  radiance  is  a  function  of  illu¬ 
mination  conditions,  scene  composition  (geometry  and  reflectance  prop¬ 
erties),  and  the  observation  geometry.  Given  that  these  factors  do 
not  remain  constant  between  frames,  variations  in  pixel  values  will 
persist  after  corrections  are  made  for  alignment  of  the  data. 

The  next  question  centers  on  how  frame  to  frame  alignment  can  be 
achieved.  There  are  two  basic  approaches  to  this  problem  which  can  be 
combined  as  appropriate.  The  first  approach  is  to  align  the  data 
"open  loop'  by  measuring  the  pointing  geometry  of  the  sensor  to  the 
accuracy  determined  by  an  error  analysis.  The  second  approach  is  to 
align  frames  "closed  loop"  by  examining  the  image  data  for  frame  to 
frame  correlations.  Selecting  the  manner  in  which  these  approaches 
are  combined  involves  an  analysis  of  the  cost,  complexity,  and  perfor¬ 
mance  trade-offs.  We  assume  that  some  "closed  loop"  correction  will 
be  necessary.  Reasonable  bounds  on  the  amount  and  type  of  displace¬ 
ment  that  can  be  expected  after  open  loop  corrections  will  guide  the 
effort  employed  in  closed  loop  operations.  Closed  loop  corrections 
would  be  based  upon  contrast  differences  in  the  imagery.  The  number 
and  spatial  accuracy  of  corrections  derived  from  contrast  variations 
needed  for  achieving  the  desired  system  performance  can  be  estimated 
from  the  error  analysis.  Some  preliminary  observations  should  be  of 
some  interest.  Suppose  the  overlap  of  the  120  pixels  image  is  between 
5%  and  10%  (6-12  lines  of  data).  For  a  nadir  orientation,  this  yields 
9  to  18  inches  of  overlap  at  200  meters!  The  numbers  quadruple  for  30 
degree  depression  angle,  but  the  features  are  still  very  small.  With 
only  a  six  to  twelve  pixel  strip  for  matching,  feature  matching  tech¬ 
niques  will  be  starved  for  data.  Correlation  techniques  therefore 
will  be  very  sensitive  to  the  variations  in  image  formation 
phenomenology. 
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On  the  question  of  measurement  accuracy,  a  common  procedure  for 
geometric  correction  involves  a  trade-off  between  sensor  position  and 
ground  position  information.  The  scenario  typically  proceeds  as  fol¬ 
lows.  The  nominal  area  being  imaged  is  determined  from  sensor  posi¬ 
tion  and  scan  data.  An  analyst  uses  this  information  to  determine 
what  ground  data  to  use  for  correlation  with  the  imagery,  e.g.,  what 
map  to  select.  Corresponding  points  in  the  ground  data  and  imagery 
are  determined.  Using  the  ground-based  positional  data,  the  original 
imagery  is  reprojected  to  a  more  nearly  constant  and  geometrically 
correct  pixel  grid.  In  short,  the  ground  position  information  is 
considered  to  be  more  accurate  than  some  of  the  sensor  position  and/or 
scan  parameters.  The  accuracy  attained  by  sensor  modelling  (FOV, 

IFOV,  scan  rate,  depression  angle,  etc.)  usually  results  in  ground 
position  data  being  substituted  for  sensor  position  data.  Our  under¬ 
standing  of  ASETS  and  CASH  data  is  that  no  ground  based  positional 
information  will  be  available.  As  such,  the  geometric  accuracy  of  the 
image  data  will  be  solely  dependent  upon  the  accuracy  of  sensor  posi¬ 
tion  and  modelling  parameters. 

The  1 ine-of-sight  origin  is  typically  determined  through  INS 
equipment,  measurement  of  the  mounting  platform,  the  modelling  of  the 
sensor  scan  characteristics.  To  minimize  flexion  and  displacement 
effects,  INS  equipment  is  rigidly  mounted  as  close  as  possible  to  the 
sensor.  A  good  understanding  of  the  frequency  of  perturbations  is 
needed  to  guide  the  sampling  rate  of  INS  values.  In  this  case,  the 
sensor  is  mounted  on  an  active  pod  which  is  designed  to  keep  orienta¬ 
tion  errors  below  one  milliradian  with  a  one  kilohertz  bandwidth.  As 
such,  a  CASH  with  a  frame  time  of  22  mi li seconds  may  undergo  several 
positional  adjustments  in  a  single  frame.  Because  of  this  high  fre¬ 
quency,  the  potential  exists  for  coupling  effects  between  sensor  data 
and  pod  corrections.  An  error  analysis  is  needed  to  determine  the 
significance  of  these  effects. 

Given  an  adequate  understanding  and  measurement  of  the  sensor 
pointing  geometry,  reprojection  of  the  data  to  a  constant  grid  size  is 
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possible.  For  frame  to  frame  alignment,  subpixel  accuracy  is  needed 
for  the  image  seam  to  be  imperceptible.  Thus,  while  it  may  be  pos¬ 
sible  to  correct  data  to  an  identical  and  constant  pixel  size,  there 
is  no  guarantee  that  interframe  pixels  will  be  aligned.  Under  optimal 
conditions,  a  single  corresponding  location  from  each  frame  will  suf¬ 
fice  to  shift  the  images  into  alignment.  More  realistically,  several 
corresponding  pixels  must  be  determined  and  their  locations  fitted  to 
a  polynomial.  This  polynomial  is  then  used  to  bring  one  image  into 
alignment  with  the  other.  Determination  of  corresponding  locations 
between  frames  presents  several  challenges.  In  particular,  the 
regions  should  be  small  (one  pixel),  unambiguous  in  correlation,  and 
sufficient  in  number  and  orientation  so  as  to  span  the  dimensional 
space  of  the  alignment  correction. 
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3.0  ERROR  ANALYSIS  FRAMEWORK 

The  goal  of  this  section  is  to  develop  the  equations  that  relate 
the  errors  in  pixel  location  to  the  various  sources  of  error.  By 
using  a  small  error  approximation  the  covariance  of  the  pixel  location 
errors  is  computed  from  the  covariance  of  the  sources  errors. 

The  first  step  is  to  obtain  an  expression  of  the  intersection  of 
the  sensor's  line-of-sight  with  the  ground.  This  is  formed  by  premul¬ 
tiplying  the  aircraft  INS  position  relative  to  a  ground  coordinate 
frame  by  a  series  of  translational  and  rotational  transformations. 

The  resultant  intersection  point  is  in  ground  coordinates. 

The  aircraft  INS  position  relative  to  the  ground  coordinate  frames 
is 

T  =  Xj(t)i  +  yj(t)j  +  Zj(t)k 

A  new  coordinate  frame  about  the  INS,  aligned  with  the  ground 
frame,  is  then: 

1  0  0  Xj(t) 

0  1  0  yj(t) 

0  0  1  Zj(t) 

0  0  0  1 

The  attitude  of  this  frame  varies  in  time  as  the  airplane  pitches 
(^Iy(^))'  (5iz(t)),  and  rolls  (5ix(^))*  I  is  rotated  by 

premultiplying  by  rotational  transformations  for  each  of  these 
motions,  and  they  are: 
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rot  ^ 


rot  = 


rot  ■ 
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0 

-sin 

0 

cos 

sin 

0 

0 

0 


1 

0 


sin  5jy(^) 


cos  5jy(^) 
0 


cos 

sin 


sin  0 

cos  ® 

0  1 

0  0 

0 

-sin 

cos 

0 


0 

0 

1 

0 

0 

0 

IJ 

0 

0 

0 

1 


pitch 


yaw 


roll 


The  INS  frame  position  as  a  function  of  time  is  thus  described  by 
the  product: 

I  =  rot  ^iY(t)  rot  ^iz(t)  rot  ^ix(t)  Iq. 

The  gimbal  center  of  the  sensor  turret  is  assumed  to  be  positioned 
rigidly  remote  from  the  INS  by  a  vector 

_  ys  yN  ^ 

R  =  x^i  +  y^j  +  Zjk 


A  frame  at  the  sensor  turret  gimbal  center  is  obtained  by  pre¬ 
multiplying  frame  I  by  a  translational  transformation  as  follows: 
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T  = 


1 

0 

0 
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0 

1 

0 
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0 

0 

1 

0 


X 


s 


I 
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Frame  T,  the  turret,  is  servoed  in  three  orthogonal  axes:  azi¬ 
muth,  Bjiit) ,  elevation  rotational 

transformations  for  these  are: 


COS 

-sin 

ejz(t)  0 

O' 

sin 

cos 

0 

0 

rot 

0 

0  1 

0 

0 

0  0 

1. 

cos  5jY(t) 

0 

sin  5yY(t) 

O' 

0 

1 

0 

0 

rot 

«jY(t)  = 

-sin 

0 

cos  ^YY^^) 

0 

0 

0 

0 

1. 

1  0 

0 

0' 

0  cos  djy^ 

(t) 

-sin 

0 

rot 

0  sin 

(t) 

cos  ^YX^^^ 

0 

.0  0 

0 

1. 

The  turret  frame  is  premultiplied  by  these  transformations,  giving 
a  new  turret  frame  S  (for  sensor,  which  is  rigidly  affixed  to  the 
turret) 

S  =  rot  flTx(^)  ’"0^  ®TY(t)  ^TX(^) 
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Within  the  sensor,  a  rotating  optic  sweeps  out  a  scanning  IFOV. 
The  axis  of  rotation  is  parallel  to  the  turret  Z-axis.  The  sensor 
frame  S  is  premultiplied  by  a  rotational  transformation  to  yield  a 
frame  the  X-axis  of  which  is  the  sensor's  1 ine-of-sight. 


cos  SgCt)  -sin  ^^(t)  0  0 


sin  ^^(t) 

0 

0 


cos  9^{t)  0  0 

0  1  0 

0  0  1. 


The  intersection  of  the  X-axis  of  frame  V  and  the  ground  is  the 
point  at  which  the  sensor  looks.  The  ground  plane  is  described  by; 
1=0  and  has  a  normal  unit  vector 

The  origin  of  frame  V  is  at:  (V14,  V24,  V34) . 

The  direction  of  frame  Vs  X-axis  is  described  by  the  unit  vector 

Viii  +  V2jj  +  V3jk 


From  this  information  the  parametric  equations  of  a  line  along 
frame  Vs  X-axis  are  formed: 


^  -  ^4  y  -  ^24  ^  -  ^34 

''11  ''21  ^31 


t 


Rearranging, 


X 

y 

z 


t  +  V 
t  +  V 
t  +  V 


14 

24 

34 


Substituting  the  equations  of  the  line  along  frame  Vs  X-axis  into  the 
equation  for  the  ground  plane; 
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t  +  V 


t  = 


Substituting  this  result  back  into  the  parametric  X-axis  line 
equations  yields  the  intersection  of  the  sensor  1 ine-of-sight  and  the 
ground  plane: 


r-v 


'11 


34 


'31 


r-v 


+  V 


14'  *21 


34 


'31 


r-v 


+  V 


24'  *31 


34 


.  '^31. 


+  V 


34 


The  actual  ground  point  is  some  distance  h  above  the  ground  plane 
(h  can  be  negative,  of  course).  The  resulting  actual  intersection 
point  is  displaced  in  x  and  y  of  the  ground  coordinate  frame.  To  find 
these  displacements,  the  first  step  is  to  find  the  angle  between  the 
X-axis  and  frame  V  and  the  ground  plane  normal  using  the  dot  product: 


cos  a 


Vlli  *  V2ij  ^  ^ 

— 2  2  ^ 

J''Il  "  ^21  ^  ''Sl 


The  angle  between  the  frame  V  x-axis  and  the  ground  plane  itself 
is  90®  -  a.  The  triangle  formed  by  90*  -  a,  h,  and  the  ground  plane 
has  a  hypotenuse  t,  where  t  is  the  distance  along  the  1 ine-of-sight 
between  the  actual  ground  point  and  the  intersection  with  the  ground 
plane.  It  is  formed  into  a  vector  by  multiplying  the  frame  V  X-axis 
unit  vector  by  and  this  vector  is  subtracted  from  the  ground  plane 
intersection  point. 


r-v 


=  V 
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+  V 


14 


Jl  V 

Vji  ''ll 


11 


'21 


'34 


'31. 


+  V 


24 


—  V 
V31  ''21 


Rearranging: 


.hi 

-  hi 

14 

''31 

r34  "J 

hi 

fv...  -  hi 

24 

CO 

> 

1 

r34 

Calling  this  result  X  =  (x',  y'),  the  objective  is  to  determine 
the  covariance  of  X  given  X  is  a  function  of  all  the  variables 
encountered  in  the  transformation.  They  form  the  vector: 

P  =  [h(t),  B^^it).  Bj^(t),  e^j^(t),  fijY(t)* 


Starting  with; 

Cov  X(p)  =  E 


x(p)  -  e{x(p)}  X(p)  -  e{x(p)} 


expand  X(p)  in  a  Taylor  series; 


X(p)  =  X(p) 


Vp[x(p)]  I  [p  -  pj 


P=P, 


P=P, 


higher  order  terms 
(assume  neglegible) 


the  choosing  Po  =  E{p} 


e{x(p)}  =  x[pj  +  vp[x(pT] 


T 


P=Po 
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e{x(p)}  =  x(pj 

Substituting  this  result  into  the  equation  for  Cov  X(p): 

» 

Cov  X(p)  -  E  X(pj  t  Vp[x(p)]^  _  _  [P  -  Po]  -  *[Po] 

I  P-Po 


Next,  each  of  the  components  of  the  above  equation  are  formed. 


''if  ’P  ''31  *  ''ii''i}  ''P  ''34  -  ''P" 

''if  ''P  ''31  *  ''2i''il  'P  ''34  -  ''2l''il  ''P'’ 


13 


''34k  ■  000000000  0) 
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where  the  VpV^ji^  terms  are  obtained  by  taking  the  partial  derivative 
of  the  V  matrix  with  respect  to  all  the  elements  of  Ji.  Recall  that 
matrix  V  is  the  product  of  nine  matrices,  and  p  is  a  vector  of  eleven 
elements. 

Pj  =  h(t),  P2  =  ^^(t).  P3  =  5^(t).  P4  =  d^it).  Pg  = 

^6  ~  '  ^8  ”  '  Pg  ~  Pjo  *  ^11  * 
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-sin 

cos  e^^Ct) 

0 

0 

-sin  ^yyU) 
0 

-cos  0jY(t) 
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0 

0 

0 

cos  0j^(t) 

-sin 

0 

0 

sin  0j^(t) 

cos  0Yx(t) 

0 

0 

0 

0 

1. 

1 

0  0 

"s 

0 

1  0 

0 

0  1 

"s 

0 

0  0 

1 . 

cos 

0  sin  fijY^ 

0 

0 

1  0 

1 

-sin  ^jyC^) 

0  cos 

0 

0 

0  0 

1 

cos 

-sin  ®j2(t)  0 

0 

sin 

cos  ® 

0 

0 

0  1 

0 

0 

0  0 

1 

•1 

0 

0 

0 

0 

cos 

-sin 

0 

0 

sin  ffjy(t) 

cos  ffry(t) 

0 

3  -sin  -cos  0 

3  cos  ffjyCt)  -sin  0 


-sin  0  cos  dTw(t)  0 


-cos 

0 

-sin 

cos 

0 

0 

D  0 


0  0  0 

0  -sin  ® 

0  0  0 

-cos  ® 

-sin  ®  ® 


-sin 

cos  5Tv(t) 


-cos  Sj^(t) 
-sin  5Tv(t) 


^ERIM 


1 

0 

0 

Xj(t)‘ 

0 

0 

0 

1  ■ 

0 

1 

0 

yAt) 

0 

0 

0 

0 

9Pq 

0 

0 

0 

0 

0 

0 

1 

2  At) 

1 

.0 

0 

0 

0. 

0 

0 

0 

1  J 

0 

0 

0 

0 

0 

0 

0 

O' 

8Mg 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

apu 

0 

0 

0 

1 

.0 

0 

0 

0. 

0 

0 

0 

0. 

All  the 

other 

partial  derivatives  of  the  M's 

wi 

th 

respect  to 

the  p's  are  all  zeroes.  Using  the  above  matrices, 
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VpV^ 


9V 

apj 


8V 


ap 


llJ 


aM/ 

— ^ 

l9Pi 


^j+1  •  •  •  ^9 


k  =  1 


^j+1  •  •  •  ^91 


VpV^  is  a  [44  x  4]  matrix  that  has  been  triple  indexed  as  an  [11  x  1] 
where  each  of  the  eleven  elements  is  a  [4  x  4].  The  equation  for 
Vp 
V. 


Vp^X(p)  on  page  10  then  uses  elements  of  VpV^  as  well  as  elements  of 


The  next  component  of  Cov  X(p)  to  be  formed  is 

,TI 


[p  -  Po]  [p  -  Po] 
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Because  the  interest  is  in  how  the  variables  of  p  correlate  in 
time,  the  above  will  be  written  as: 

T1 

E'  [Ap{tj)][A?(t2)]  =  E{Ap.(tj)Apj(t2)}  . 

ft  j  V 

This  results  in  a  [11  x  11]  matrix  that  is  strongly  diagonally 
dominant  and  symmetric.  Values  for  the  nonzero  elements  of  this 
matrix  are  then  to  be  taken  from  models  of  the  actual  random  processes 
describing  the  behavior  of  the  IJ  variables  with  time. 

The  final  result,  Cov  X(p),  is  dimensionally: 

r  1  [11 

Cov  X(p)  =  [2  X  11]  [11  X  llj  X 
=  [2x2] 


